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Executive  Summary 


Although  a  considerable  amount  of  research  has  focused 
on  the  relationship  between  sector  characteristics  and 
controller  workload  or  perceived  complexity,  relatively 
few  studies  have  examined  the  relationship  between 
sector  characteristics  and  the  occurrence  of  operational 
errors  (OEs).  In  many  early  studies  of  OE  causal  factors, 
examinations  of  sector  characteristics  were  limited  to 
purely  theoretical  relationships  or  to  traffic  counts  and 
altitude  transitions  of  the  involved  aircraft.  When  sector 
characteristics  were  examined,  they  consisted  primarily 
of  ratings  rather  than  objective  measures.  Most  of  this 
work  has  been  conducted  without  reference  to  routine 
operations  (ROs).  Yet,  for  every  OE  that  occurs  in  a 
sector,  there  are  hundreds  (possibly  thousands)  of  hours 
in  which  an  OE  did  not  occur.  To  truly  understand  the 
environmental  and  contextual  factors  that  contribute  to 
OEs,  it  is  necessary  to  identify  what  was  different  about 
the  sector  environment  at  the  time  the  OE  occurred. 

In  the  present  study,  OE  and  RO  traffic  samples  were 
compared  using  logistic  regression  analysis.  Two  separate 
logistic  regression  analyses  were  conducted  for  low-  and 
high-altitude  sectors.  The  OE  sample  was  drawn  from 
119  OEs  occurring  in  the  Indianapolis  en  route  airspace 
from  9/17/2001  through  12/10/2003.  Of  these,  40  oc¬ 
curred  in  the  low-altitude  sectors  and  79  occurred  in 
the  high-altitude  sectors.  The  RO  traffic  samples  were 
recorded  between  5/8/2003  and  5/10/2003.  These 
data  were  processed  in  5-minute  intervals  using  custom 
software  designed  to  calculate  objective  measures  from 
routinely  recorded  National  Airspace  System  (NAS)  data. 
This  produced  a  total  of  2763  RO  traffic  samples.  The 
79  high- altitude  OE  traffic  samples  were  combined  with 
79  randomly-selected  high- altitude  RO  traffic  samples 
to  produce  a  total  of  158  traffic  samples  for  the  high- 
altitude  sector  analysis 

In  logistic  regression  analysis  of  the  low- altitude  sec¬ 
tor  samples,  Average  Control  Duration,  the  Number  of 
Handoffs,  the  Number  of  Heading  Changes,  the  Number 
of  Intersecting  Flight  Paths,  the  Number  of  Point  Outs, 
and  the  Number  ofTransitioning  Aircraft  were  submitted 
as  the  initial  set  of  predictors.  Backward  stepwise  elimina¬ 
tion  reduced  the  variables  in  the  final  low- altitude  sector 
model  to  the  Number  of  Intersecting  Flight  Paths,  the 
Number  of  Point  Outs,  and  the  Number  of  Handoffs. 
In  the  low- altitude  sector  model,  each  intersecting  flight 
path  increased  the  likelihood  that  the  traffic  sample  was 
an  OE  by  1 89%,  each  point  out  increased  the  likelihood 
by  57%,  and  each  handoff  increased  the  likelihood  that 
the  traffic  sample  was  an  OE  by  19%.  Of  the  40  ROs  in 
the  low- altitude  sample,  32  (80%)  were  correctly  classified 


and  8  (20%)  were  misclassified  as  OEs.  Of  the  40  OEs 
in  the  sample,  28  (70%)  were  correctly  classified  and  12 
(30%)  were  misclassified  as  ROs.  Overall,  the  low- altitude 
model  had  75%  classification  accuracy. 

In  logistic  regression  analysis  of  the  high- altitude  sec¬ 
tor  samples,  Average  Control  Duration,  the  Number  of 
Handoffs,  the  Number  of  Heading  Changes,  the  Number 
of  Intersecting  Flight  Paths,  the  Number  of  Point  Outs, 
and  the  Number  ofTransitioning  Aircraft  were  submit¬ 
ted  as  the  initial  set  of  predictors.  Backward  stepwise 
elimination  reduced  the  variables  in  the  final  high-alti¬ 
tude  sector  model  to  the  Number  of  Intersecting  Flight 
Paths,  the  Number  of  Heading  Changes,  the  Number  of 
Transitioning  Aircraft,  and  Average  Control  Duration. 
In  the  high-altitude  sector  model,  each  one-unit  increase 
in  the  Number  of  Intersecting  Flight  Paths  increased  the 
likelihood  that  a  traffic  sample  was  an  OE  by  1 00%,  each 
one-unit  increase  in  the  Number  of  Heading  Changes 
increased  the  likelihood  by  36%,  every  Transitioning 
Aircraft  increased  the  likelihood  by  27%,  and  each  one- 
second  increase  in  Average  Control  Duration  increased 
the  likelihood  by  1%.  Of  the  79  ROs  in  the  high-altitude 
sample,  64  (81%)  were  correctly  classified  and  15  (19%) 
were  misclassified  as  OEs.  Of  the  79  OEs  in  the  sample, 
60  (76%)  were  correctly  classified  and  19  (24%)  were 
misclassified  as  ROs.  Overall,  the  high-altitude  model 
had  79%  classification  accuracy. 

The  results  of  the  logistic  regression  analyses  indicate 
that  sufficient  models  may  be  constructed  from  sector 
characteristics  variables.  Overall  classification  accuracy  be¬ 
tween  75-7 9%  is  remarkable  for  models  constructed  solely 
of  environmental  and  contextual  factors.  After  all,  other 
factors  (e.g.,  human  elements,  organizational  influences) 
also  contribute  to  the  occurrence  of  OEs.  Unfortunately, 
all  the  logistic  regression  models  were  better  at  classifying 
ROs  than  OEs.  Classification  of  OEs  ranged  from  as  low 
as  70%  in  the  low- altitude  sector  sample  to  76%  in  the 
high-altitude  sample.  Although  this  level  of  accuracy 
would  be  unacceptable  for  most  automation  tools,  it  is 
unrealistic  to  expect  definitive  results  from  one  or  two 
analyses.  Moreover,  the  sector  characteristic  variables  used 
in  these  analyses  do  not  represent  an  exhaustive  list  of  all 
the  potential  predictors  of  OEs. 

Although  logistic  regression  cannot  be  used  to  identify 
causal  factors  directly  (i.e.,  prediction  is  not  the  same  as 
causation),  the  logistic  regression  coefficients  do  provide 
information  about  the  likelihood  of  an  OE  relative  to  the 
predictors  in  the  model.  Thus,  the  results  have  immedi¬ 
ate  heuristic  value  in  that  they  invite  questions  about 
how  the  dynamic  predictors  interact  with  static  sector 


characteristics.  Dynamic  elements  lend  themselves  to 
automation  applications,  but  static  characteristics  must  be 
addressed  for  sector  restructuring.  The  dynamic  predictors 
that  make  up  the  logistic  regression  models  are  indicants 
of  conditions  that  discriminate  between  OEs  and  ROs. 
These  indicants  may  be  used  to  reveal  aspects  of  the  sector 
environment  that  might  be  altered  to  reduce  the  number 
of  OEs.  For  example,  the  combination  of  the  Number 
of  Point  Outs  and  the  Number  of  ffandoffs  in  the  low- 
altitude  sector  model  may  indicate  that  the  location  of 
sector  boundaries  increases  coordination  workload  and 
complexity.  On  the  other  hand,  the  combination  of  the 
Number  of  Point  Outs  and  the  Number  of  Intersecting 
Flight  Paths  may  point  to  problems  with  the  orientation 
of  traffic  paths  relative  to  those  boundaries.  The  com¬ 
bination  of  the  Number  of  Heading  Changes  and  the 


Number  of  Transitioning  Aircraft  in  the  high-altitude 
sectors  is  suggestive  of  traffic  complexity  in  high-altitude 
sectors  adjacent  to  low- altitude  arrival  or  departure  sec¬ 
tors.  Average  Control  Duration  may  be  a  function  of  the 
size  of  high-altitude  sectors. 

Given  the  sample  size  and  consequent  restriction  of 
the  predictor  set,  there  is  no  guarantee  that  these  results 
will  generalize  to  other  samples.  Multiple  studies,  with 
samples  sizes  that  allow  for  a  more  inclusive  list  of  predic¬ 
tors,  must  be  conducted  at  a  number  of  facilities  before 
such  models  might  be  reliable  enough  for  practical  ap¬ 
plications.  Nevertheless,  the  methodology  of  comparing 
OE  and  RO  traffic  samples  is  promising.  Continued 
investigations  along  these  lines  may  highlight  complexity 
factors  that  must  be  addressed  to  ensure  that  separation 
is  maintained. 


Logistic  Regression  Analysis  of  Operational  Errors  and  Routine 
Operations  Using  Dynamic  Sector  Characteristics 


In  the  current  Air  Traffic  Control  (ATC)  system,  an 
Operational  Error  (OE)  occurs  when  there  is  a  violation  of 
aircraft  separation  minima  as  defined  by  Federal  Aviation 
Administration  regulations  (FAA,  2007).  Although  the 
FAA  has  called  for  a  reinvention  of  the  National  Airspace 
System  (NAS),  it  is  reasonable  to  assume  that  future 
systems  will  also  include  some  kind  of  aircraft  separation 
standards.  As  changes  to  procedures  and  airspace  structure 
are  being  considered,  it  becomes  increasingly  important 
to  understand  the  environmental  and  contextual  factors 
(often  referred  to  as  sector  characteristics)  that  contrib¬ 
ute  to  the  loss  of  separation  to  ensure  that  safety  is  not 
compromised. 

A  considerable  amount  of  research  has  focused  on  the 
relationship  between  sector  characteristics  and  controller 
workload  or  perceived  complexity  (e.g.,  Buckley,  De- 
Barysche,  Hitchner,  &  Kohn,  1 983;  Chatterji  &  Sridhar, 
200 1 ;  Christien  &  Benkouar,  2003;  Kopardekar  &  Mag- 
yarits,  2003;  Laudeman,  Shelden,  Branstrom,  &  Brasil, 
1998;  Mogford,  Murphy,  &  Guttman,  1 994;  Robertson, 
Grossberg,  &  Richards,  1979).  However,  relatively  few 
studies  have  examined  the  relationship  between  sector 
characteristics  and  the  occurrence  of  OEs.  In  many  early 
studies  of  OE  causal  factors,  examinations  of  sector  char¬ 
acteristics  were  limited  to  purely  theoretical  relationships 
(e.g.,  Arad,  1964;  Schmidt,  1976)  or  to  traffic  counts  and 
altitude  transitions  of  the  involved  aircraft  (e.g.,  Kershner, 
1968;  Schroeder,  1982;  Spahn,  1977). 

Grossberg  (1989)  expanded  on  this  by  collecting 
ratings  from  97  controllers  and  supervisors  regarding 
various  aspects  of  the  sector  environment.  He  found 
that  the  highest-rated  sector  complexity  factors  in  the 
Chicago  Air  Route  Traffic  Control  Center  (ARTCC) 
were  control  adjustments  (e.g.,  merging,  spacing,  and 
speed  changes),  climbing  and  descending  flight  paths, 
and  mix  of  aircraft  types.  An  index  based  on  these  factors 
was  significantly  correlated  (r  =  .74)  with  the  number  of 
OEs  occurring  in  27  Chicago  sectors  over  2 1  months  in 
1987  and  1988. 

Rodgers,  Mogford,  and  Mogford  (1998)  evaluated 
the  relationship  between  sector  characteristics  and  the 
incidence  of  OEs  at  the  Atlanta  ARTCC.  OE  data  were 
extracted  from  Facility  Operations  and  Administration 
reports  for  85  OEs  occurring  over  a  three-year  period 
from  1992  to  1995.  Sector  characteristics  variables  were 
derived  from  a  number  of  sources  and  included  static  sec¬ 
tor  characteristics  from  the  facility’s  Adaptation  Control 


Environmental  Systems  (ACES),  average  complexity  and 
density  ratings  drawn  from  the  1995  annual  facility  review, 
and  complexity  factor  ratings  provided  by  one  Airspace 
and  Procedures  specialist  from  each  area  in  the  facility. 
Several  statistical  techniques  were  employed  to  predict 
OE  occurrence  in  the  Atlanta  sectors.  In  a  linear  multiple 
regression  analysis,  ratings  of  radio  frequency  congestion 
and  the  effects  of  restricted  airspace  accounted  for  31% 
of  the  variance  in  the  number  of  OEs  per  sector.  The 
same  two  predictors  generated  a  function  that  classified 
OE  frequency  groups  (i.e.,  sectors  with  no  OEs,  fewer 
than  four  OEs,  and  sectors  with  four  or  more  OEs)  with 
58%  accuracy  in  a  discriminant  function  analysis.  Ratings 
of  weather  influences,  radio  frequency  congestion,  and 
complexity  were  significantly  different  in  a  MAN  OVA  of 
OE  frequency  groups.  The  authors  stated  that  the  overall 
Hotellings  A(16,  68)  =  1.60 ,p  =  .094  of  the  MANOVA 
was  acceptable  due  to  the  exploratory  nature  of  the  study 
and  emphasis  on  data  exploration.  Sector  size  failed  to 
emerge  as  a  significant  contributor  in  any  of  these  mod¬ 
els,  but  a  t-test  between  the  combined  no-  and  low-OE 
groups  and  the  group  consisting  of  sectors  with  four  or 
more  OEs  was  statistically  significant  (/><.05). 

In  both  the  Grossberg  (1989)  and  Rodgers,  Mogford, 
and  Mogford  (1998)  studies,  sector  characteristics  were 
evaluated  against  the  number  of  OEs  per  sector.  The  risk 
of  OE  occurrence  increases  as  a  function  of  the  number 
of  hours  a  sector  is  in  operation  (i.e.,  exposure  increases 
the  risk  of  occurrence).  If  OE  incidence  is  not  adjusted 
for  this,  the  internal  validity  of  the  analysis  may  be  com¬ 
promised.  However,  there  is  no  evidence  to  suggest  that 
these  adjustments  were  made  in  either  study. 

Another  disadvantage  of  OE  incidence  as  a  dependent 
variable  is  that  it  restricts  the  number  of  independent 
variables  that  may  be  analyzed  without  detriment  to 
external  validity.  With  most  statistical  analyses,  the  risk 
that  results  will  fail  to  generalize  increases  as  the  number 
of  independent  variables  approaches  the  number  of  cases. 
In  spite  of  this,  Rodgers,  Mogford,  and  Mogford  (1998) 
submitted  28  predictors  for  stepwise  linear  multiple 
regression  analysis  of  OE  incidence  in  45  sectors  at  the 
Atlanta  ARTCC  (i.e.,  a  ratio  of  1 .6  cases  for  each  predic¬ 
tor).  A  correlation  matrix  of  the  predictors  revealed  26 
comparisons  with  r  >  .60.  The  authors  relied  on  tests 
for  tolerance  in  the  stepwise  procedure  to  eliminate 
redundancies  and  reduce  the  number  of  variables  in 
the  predictor  set.  However,  in  stepwise  linear  multiple 
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regression,  a  minimum  of  40  cases  for  each  predictor  is 
necessary  to  ensure  the  solution  will  generalize  to  other 
samples  (Tabachnick  &  Fidell,  2006). 

Perhaps  the  biggest  drawback  to  OE  frequency  as  a 
dependent  measure  is  that  it  affords  no  comparison  with 
routine  operations  (ROs).  Yet,  for  every  OE  that  occurs 
in  a  sector,  there  are  hundreds  (possibly  thousands)  of 
hours  during  which  an  OE  did  not  occur.  Variables  that 
correlate  with  sector  OE  frequency  do  not  describe  what 
was  different  about  the  sector  at  the  time  of  the  OE. 
To  truly  understand  the  environmental  and  contextual 
factors  that  contribute  to  OEs,  it  is  necessary  to  identify 
what  was  different  about  the  sector  environment  at  the 
time  the  OE  occurred. 

Pfleiderer  and  Manning  (2007)  conducted  an  investi¬ 
gation  to  determine  whether  logistic  regression  analysis 
of  objective  sector  characteristics  could  discriminate 
between  OE  and  RO  traffic  samples.  Prior  to  this  study, 
investigations  of  the  contribution  of  sector  characteristics 
to  the  occurrence  of  OEs  relied  heavily  on  attributed 
causal  factors  and  subjective  ratings.  Although  interviews 
and  ratings  about  the  importance  of  complexity  factors 
are  beneficial  in  the  early  stages  of  research,  practical 
prediction  models  must  eventually  be  calculated  from 
objective  measures.  After  all,  it  is  the  actual  characteristics 
of  the  sectors  that  must  be  addressed  when  developing 
strategies  to  reduce  OEs  rather  than  opinions  or  beliefs 
about  those  characteristics.  Two  separate  logistic  regres¬ 
sion  analyses  were  performed  for  high-  and  low- altitude 
sector  samples  at  the  Indianapolis  ARTCC  (ZID).  In 
the  high-altitude  sector  sample,  a  logistic  regression 
model  comprising  the  Number  of  Heading  Changes,  the 
Number  of  Transitioning  Aircraft,  and  Average  Control 
Duration  was  able  to  accurately  classify  80%  of  the 
OE  and  RO  traffic  samples.  In  the  low- altitude  sector 
sample,  variables  included  in  the  final  model  were  the 
Number  of  Point  Outs,  the  Number  of  Handoffs,  and 
the  Number  of  Heading  Changes.  This  model  was  able 
to  accurately  classify  79%  of  the  low- altitude  OE  and 
RO  traffic  samples. 

Unfortunately,  the  Pfleiderer  and  Manning  (2007) 
study  was  flawed.  Available  traffic  data  consisted  of  OEs 
from  9/17/200 1  to  1 2/ 1 0/2003  and  ROs  from  2/25/2005 
to  3/3/2005.  Clearly,  the  time  differential  between  the 
OE  and  RO  traffic  samples  was  a  confounding  influence 
because  it  represented  an  uncontrolled,  systematic  dif¬ 
ference  between  the  two  groups. 

A  second  problem  with  the  Pfleiderer  and  Manning 
(2007)  design  involved  pairing  OE  and  RO  traffic  samples 
(by  sector,  day  of  week,  and  time  of  day).  Logistic  re¬ 
gression  analysis  assumes  that  all  cases  are  independent 
of  one  another.  Violation  of  this  assumption  results  in 
over-dispersion,  which  produces  an  inflated  Type  I  error 


rate  for  tests  of  predictors  (Tabachnick  &  Fidell,  2006). 
Although  there  is  no  reason  to  assume  that  errors  would 
be  correlated  between  the  OE  and  RO  traffic  samples 
(particularly  in  light  of  the  disparate  time  frames  from 
which  the  samples  were  drawn),  neither  is  there  evidence 
to  the  contrary.  Random  selection  of  RO  traffic  samples 
would  have  guaranteed  that  the  assumption  of  indepen¬ 
dence  had  been  met. 

Another  possible  source  of  inflation  for  tests  of  predic¬ 
tors  involved  the  variables  Number  of  Heading  Changes 
and  Number  of  Transitioning  Aircraft.  When  controllers 
become  aware  that  an  OE  is  developing,  they  may  issue 
clearances  in  an  attempt  to  resolve  the  situation.  Inclu¬ 
sion  of  altitude  or  heading  changes  made  in  response  to 
such  clearances  would  inflate  the  contributions  of  the 
Number  of  Transitioning  Aircraft  (based  on  the  number 
of  altitude  changes)  and  the  Number  of  Heading  Changes 
as  predictors  of  OEs.  The  choice  of  processing  interval 
(four  minutes  prior  to  loss  of  separation  and  one  minute 
after)  may  have  magnified  this  effect. 

In  the  present  study,  OE  and  RO  traffic  samples  are 
again  compared  using  logistic  regression  analysis,  but  some 
important  modifications  were  made  to  the  design.  Both 
the  OE  and  RO  traffic  samples  used  to  build  the  logistic 
regression  model  were  from  2003,  thus  eliminating  the 
confound  of  the  previous  analysis.  A  randomly-selected 
sub-sample  of  the  RO  traffic  data  was  designated  for 
model  building.  The  remaining  RO  cases  and  traffic 
samples  from  OEs  occurring  in  the  ZID  airspace  from 
2001-2002  were  set  aside  for  cross  validation.  No  at¬ 
tempt  was  made  to  match  the  RO  traffic  samples  to  the 
OE  samples,  thus  meeting  the  assumption  of  indepen¬ 
dence  and  eliminating  the  potential  for  Type  I  errors  for 
tests  of  predictors.  The  Number  of  Heading  Changes 
and  Number  of  Transitioning  Aircraft  were  adjusted  to 
eliminate  changes  made  in  response  to  control  actions  to 
avoid  an  imminent  OE  (described  in  detail  in  the  Method 
section)  to  guard  against  inflated  contributions  by  these 
two  predictors.  In  addition,  the  processing  interval  for 
the  OE  traffic  samples  was  changed  to  include  only  the 
five  minutes  prior  to  initial  loss  of  separation. 

Unfortunately,  restricting  the  OE  traffic  samples  to 
a  single  year  reduces  the  sample  size.  This  prohibits 
separation  of  low-  and  high- altitude  sector  samples  for 
individual  analyses,  even  though  there  is  reason  to  suspect 
they  constitute  heterogeneous  sub-samples.  To  begin  with, 
OE  incidence  differs  between  strata.  Of  119  OEs  in  the 
full  sample,  40  occurred  in  low-altitude  sectors,  and  79 
occurred  in  the  high- altitude  sectors.  Inclusion  of  sector 
strata  as  a  binary  categorical  variable  would  provide  no 
information  beyond  this  fact.  Second,  separate  logistic 
regression  analyses  of  the  low-  and  high- altitude  samples 
produced  two  very  different  models  in  the  Pfleiderer  and 
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Manning  (2007)  study.  It  is  possible  that  combining 
low-  and  high-altitude  sector  samples  would  produce 
a  model  that  fits  the  high-altitude  sectors  poorly  and 
the  low-altitude  sectors  not  at  all.  Cross  validation  can 
be  used  to  determine  whether  the  2003  data  are  similar 
enough  to  the  2001-2002  data  to  justify  pooling  the 
OE  samples.  If  OE  classification  is  similar  in  the  model¬ 
building  and  cross-validation  results,  this  suggests  that 
the  model  works  equally  well  for  both  samples  and  that 
they  may  be  similar  enough  to  be  pooled.  If  that  is  the 
case,  the  OE  data  will  be  pooled,  and  separate  logistic 
regression  analyses  will  be  conducted  for  the  low-  and 
high-altitude  sector  samples. 

The  sector  characteristics  used  as  independent  variables 
in  this  study  are  based  on  measures  that  have  previously 
demonstrated  a  relationship  with  workload,  complexity, 
or  the  occurrence  of  OEs.  These  variables  are  described 
in  detail  in  the  following  paragraphs. 

Predictor  Variables 

Average  Control  Duration.  Aircraft  control  duration 
is  influenced  by  a  number  of  factors,  including  aircraft 
performance  characteristics,  Traffic  Management  Initia¬ 
tives  (TMI),  and  sector  size  -  all  of  which  have  been  as¬ 
sociated  with  sector  workload  or  complexity  (Grossberg, 
1989;  Mogford,  Murphy,  &  Guttman,  1994;  Pfleiderer, 
Manning,  &  Goldman,  2007) .  Average  Control  Duration 
is  the  mean  of  the  durations  (in  seconds)  of  all  aircraft 
controlled  by  the  sector  within  a  processing  interval. 
Control  time  occurring  before  or  after  the  interval  was 
not  included  in  the  calculations. 

Number  of Handoffs.  Although  traffic  count  remains  the 
best  single  predictor  of  the  number  of  OEs  on  a  national 
level,  previous  research  suggests  that  it  is  not  an  effective 
predictor  of  OEs  at  the  sector  level  (e.g.  Schroeder,  1982; 
Schroeder,  Bailey,  Pounds,  &  Manning,  2006;  Spahn, 
1977).  Aside  from  doubts  about  its  effectiveness  as  a  pre¬ 
dictor,  perhaps  the  biggest  drawback  to  traffic  count  is  that 
it  tends  to  be  highly  correlated  with  other  traffic-related 
measures.  As  a  result,  traffic  counts  create  redundancies 
that  may  overshadow  more  effective  predictors.  Handoffs, 
however,  are  correlated  with  the  number  of  aircraft  in 
the  sector,  but  they  are  not  synonymous  with  it.  In  the 
ZID  traffic  samples  used  in  this  study  (described  in  the 
Method  section),  the  total  number  of  aircraft  and  the 
total  number  of  handoffs  had  a  Spearman’s  correlation  of 
r  =  .66  (N  =  2763).  Handoff  counts  capture  elements  of 
communication  workload  and  required  attention.  Handoff 
initiates  (i.e.,  outbound  handoffs  from  the  current  sector 
to  another  sector  or  facility)  are  generally  associated  with 
the  issuance  of  a  frequency  change  clearance.  Handoff 
accepts  (i.e.,  inbound  handoffs  from  another  sector  or 
facility  to  the  current  sector)  are  accompanied  by  an 


eventual  (but  not  necessarily  concurrent)  verification  that 
aircraft  coming  into  the  sector  are  tuned  to  the  appropriate 
frequency.  Handoff  counts  also  provide  information  about 
coordination  and  required  procedures.  Although  most 
handoffs  are  fairly  automatic,  some  require  coordination 
with  other  sectors.  In  some  instances,  aircraft  must  com¬ 
ply  with  altitude  or  other  restrictions  before  they  can  be 
handed  off  or  a  handoff  can  be  accepted.  Handoffs  may 
also  reflect  the  impact  of  sector  geography.  According  to 
Couluris  and  Schmidt  (1973),  the  number  of  handoffs, 
coordination,  and  point  outs  “result  from,  or  are  influenced 
by,  the  existence  and  design  (shape)  of  the  sectors.  The 
additional  work  created  can  be  thought  of  as  the  cost  of 
sectorization”  (p.  657).  The  Number  of  Handoffs  is  the 
total  number  of  handoff  initiates  and  handoff  accepts 
occurring  within  the  5-minute  processing  interval. 

Number  of  Heading  Changes.  Heading  changes  have 
demonstrated  a  relationship  with  controller  ratings  of  ac¬ 
tivity  (e.g.,  Laudeman  et  al.,  1998),  workload  (e.g.,  Stein, 
1985),  and  complexity  (e.g.,  Kopardekar  &  Magyarits, 
2003).  Heading  changes  are  involved  with  a  number 
of  procedures  such  as  merging  and  spacing,  Standard 
Terminal  Arrival  Routes  (STARs),  Standard  Instrument 
Departure  Routes  (SIDs),  and  holding.  The  Number 
of  Heading  Changes  is  a  count  of  all  turns  in  excess  of 
10  degrees  per  12-second  radar  update  that  continue  in 
the  same  direction  for  at  least  three  updates.  Heading 
changes  made  in  an  attempt  to  avoid  an  imminent  OE 
were  excluded  from  the  counts  for  OE  traffic  samples. 
The  criteria  for  these  exclusions  are  described  in  the 
Method  section. 

Number  of  Intersecting  Flight  Paths.  This  factor  was 
one  of  the  highest  rated  complexity  factors  in  the  high- 
altitude  and  super  high-altitude  sectors  in  the  Pfleiderer, 
Manning,  and  Goldman  (2007)  study,  in  which  a  sample 
of  32  controllers  and  4  supervisors  from  ZID  provided 
ratings  for  a  set  of  22  sector  complexity  factors.  In  ad¬ 
dition,  the  Number  of  Intersecting  Flight  Paths  was 
associated  with  a  component  (i.e.,  composite  factor 
score)  that  demonstrated  a  reliable  relationship  with 
the  number  of  OEs  in  the  ZID  sectors.  A  similar  factor 
(several  traffic  flows  converging  at  the  same  point)  was 
among  the  top-rated  complexity  factors  in  an  investiga¬ 
tion  of  Maastricht  airspace  conducted  by  Eurocontrol 
(2006).  The  Number  of  Intersecting  Flight  Paths  is  the 
maximum  number  of  flight  paths  that  might  be  expected 
to  intersect,  irrespective  of  altitude,  within  a  1 0-minute 
projected  time  frame  given  the  current  speed  and  trajec¬ 
tory  of  the  aircraft.  Projections  were  calculated  at  each 
12-second  radar  update  within  each  minute  of  data.  The 
length  and  slope  of  the  projected  paths  were  based  on 
the  distance  and  angle  of  the  current  and  previous  radar 
position  coordinates. 
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Number  of  Point  Outs.  Coordination  between  control¬ 
lers  was  one  of  the  events  selected  by  Schmidt  (1976)  for 
his  Control  Difficulty  Index  even  though  he  considered 
it  to  be  one  of  the  most  difficult  to  process  -  with  good 
reason.  Coordination  is  not  often  recorded.  However, 
point  out  entries  represent  one  of  the  few  instances  in 
which  coordination  between  sectors  is  recorded.  The 
Number  of  Point  Outs  is  the  total  number  of  point  out 
entries  made  by  the  Radar  and  Radar  Associate  controllers 
during  the  5-minute  processing  interval. 

Number  ofTransitioningAircraft.  The  amount  of  climb¬ 
ing  and  descending  traffic  has  long  been  recognized  as 
a  contributor  to  the  difficulty  of  working  a  sector  (e.g., 
Arad,  1964).  Grossberg  (1989)  observed  that  one  of  the 
factors  most  often  identified  as  being  responsible  for  sector 
complexity  in  the  Chicago  ARTCC  was  climbing  and 
descending  flight  paths.  More  recently,  Kopardekar  and 
Magyarits  (2003)  found  that  the  number  of  descending 
aircraft  and  the  number  of  altitude  changes  greater  than 
750  feet  per  minute  both  contributed  significantly  to  the 
explanation  of  variance  in  a  linear  regression  model  of 
subj  ective  complexity  ratings  collected  at  the  Fort  Worth, 
Atlanta,  Cleveland,  and  Denver  ARTCCs.  The  Number 
ofTransitioningAircraft  represents  the  number  of  aircraft 
making  one  or  more  altitude  changes  during  the  5-minute 
processing  interval.  To  be  counted  as  a  change,  altitude 
must  increase  or  decrease  by  a  minimum  of  200  feet  per 
12-second  radar  update  and  must  continue  to  change 
in  the  same  direction  for  at  least  three  updates.  Altitude 
changes  resulting  from  last-minute  clearances  made  in 
an  attempt  to  avoid  the  OE  were  excluded  from  counts 
for  the  OE  sample.  The  criteria  for  these  exclusions  are 
described  in  the  Method  section. 

Exclusion  of  Static  Predictor  Variables 

Sector  complexity  factors  are  generally  described  in  two 
ways:  static  and  dynamic  (Mogford,  Guttman,  Morrow,  & 
Kopardekar,  1995).  The  predictor  variables  j  ust  described 
represent  dynamic  sector  characteristics  because  they 
change  over  time.  Static  sector  characteristics  are  those 
that  change  infrequently  or  not  at  all  and  are  generally 
related  to  airspace  design  (e.g.,  size  of  the  sector,  number 
of  shelves  or  tunnels).  Static  sector  characteristics  may 
vary  between  sectors,  but  they  do  not  vary  within  sectors. 
Consequently,  the  variance  of  static  variables  would  be 
seriously  limited  in  the  present  study  because  multiple 
OEs  occurred  in  many  of  the  same  sectors  in  the  sample. 
Even  if  static  sector  characteristics  were  related  to  OEs, 
it  is  unlikely  this  relationship  would  be  detected.  For 
this  reason,  predictors  were  restricted  to  dynamic  sector 
characteristics.  These  were  submitted  to  logistic  regres¬ 
sion  analysis  to  determine  the  degree  to  which  they  could 
discriminate  between  OE  and  RO  traffic  samples. 


Method 

Data  Sources 

Traffc  Samples.  All  traffic  samples  were  initially  derived 
from  System  Analysis  Recordings  (SARs)  generated  by 
en  route  Host  Computer  Systems.  The  Host  features 
data  reduction  programs  that  generate  text  reports  of 
selected  subsets  of  SAR  data.  The  information  used  to 
calculate  the  predictor  variables  was  extracted  from  Log 
and  Track  reports  produced  by  one  of  these  programs, 
the  Data  Analysis  and  Reduction  Tool  (DART).  Log 
reports  include  controller  entries  and  information  sent 
to  the  radar  display  and  the  auxiliary  text  display  (e.g., 
data  blocks  and  list  items) .  Track  reports  contain  detailed 
information  (e.g.,  altitude,  heading,  ground  speed,  and 
position)  from  the  Host  computer’s  internal  radar  track 
database. 

OE  traffic  samples  were  derived  from  Systematic  Air 
Traffic  Operations  Research  Initiative  (SATORI;  Rodgers 
&  Duke,  1993)  files.  SATORI  files  contain  reconfigured 
DART  information  (i.e. ,  log  and  track  reports  in  a  slightly 
different  format).  SAR  data  are  difficult  to  obtain  from 
facilities  because  they  require  a  prohibitive  amount  of 
storage  space.  SATORI  re-creations  require  less  space  and 
so  these  files  are  often  the  only  traffic  data  saved  after  an 
OE.  Therefore,  the  primary  constraint  on  the  size  and 
range  of  the  data  set  was  the  availability  of  SATORI  re¬ 
creations.  SATORI  data  meeting  processing  criteria  (i.e., 
five  minutes  prior  to  the  initial  loss  of  separation)  were 
only  available  for  1 19  OEs  occurring  in  the  ZID  airspace 
from  9/ 17/2001  through  12/10/2003.  OE  traffic  samples 
from  2003  (n  =  48)  were  used  for  model  building,  and 
the  remaining  OE  traffic  samples  (w  =  71)  were  used  for 
cross  validation. 

The  RO  traffic  samples  were  derived  from  ZID  SAR 
data  recorded  on  5/8/2003  (15:55  to  17:05,  18:55  to 
20:10,  and  20:50  to  22:15  ZULU),  5/9/2003  (0:00  to 
1:10  ZULU),  and  5/10/2003  (11:20  to  12:40  ZULU). 
DART  log  and  track  text  reports  were  first  encoded  into 
database  files  and  then  processed  in  5-minute  intervals 
using  custom  software  designed  to  calculate  objective 
measures  from  routinely  recorded  NAS  data.  This  pro¬ 
duced  a  total  of  2644  RO  traffic  samples.  A  randomly- 
selected  set  (n  =  48)  were  used  for  model  building,  and 
the  remaining  samples  (n  =  2596)  were  set  aside  for 
cross  validation.  Random  selection  was  accomplished 
by  first  sorting  traffic  samples  by  date,  time,  and  sector 
number,  then  assigning  each  sample  a  random  number 
(generated  by  the  Microsoft  Excel  RAND  function),  and 
then  resorting  the  samples  according  to  this  value.  The 
first  48  cases  (equal  to  the  number  of  OE  samples)  were 
assigned  to  the  model-building  sample.  A  list  of  the  ZID 
sectors  and  the  number  of  associated  traffic  samples  in  the 
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model-building  and  cross-validation  samples  is  provided 
in  Table  A1  of  Appendix  A. 

In  the  model-building  sample,  the  number  of  samples 
in  the  RO  and  OE  groups  is  equivalent.  This  is  important 
because  widely  disparate  group  size  produces  logistic  re¬ 
gression  models  that  favor  (i.e. ,  over-represent)  the  largest 
group.  Equal  group  size  also  ensures  that  classification 
accuracy  in  excess  of  50%  represents  improvement  over 
chance.  On  the  other  hand,  group  size  does  not  matter 
in  the  cross-validation  sample  because  these  cases  are  not 
used  to  build  the  model.  The  existing  model  is  simply 
being  applied  to  the  sample. 

Transitioning  Aircraft  and  Heading  Change  Adjustments 

Altitude  and  heading  changes  made  in  an  attempt  to 
avoid  an  imminent  OE  were  excluded  from  the  analysis 
using  summary  incident  narratives  from  the  Operational 
Error  Investigation  form  and  lists  of  computer-detected 
changes.  The  summary  incident  narratives  described 
events  leading  up  to  the  loss  of  separation.  The  lists  in¬ 
cluded  the  aircraft  identifier,  start  time,  stop  time,  and 
direction  of  all  computer-detected  altitude  and  heading 
changes  used  in  the  calculation  of  the  Number  of  Tran¬ 
sitioning  Aircraft  and  Number  of  Heading  Changes. 

The  time  at  which  the  controller  became  aware  of  a 
potential  OE  and  any  action  taken  to  avoid  loss  of  sepa¬ 
ration  were  usually  described  in  the  narrative.  The  time 
and  content  of  the  clearances  obtained  from  the  narrative 
were  then  compared  with  the  list  of  computer-detected 
changes  for  the  OE  traffic  sample.  If  the  listed  clearance 
matched  one  of  the  changes  (i.e.,  matched  by  aircraft 
identifier,  time,  and  direction),  it  was  marked  for  exclu¬ 
sion.  In  the  event  that  the  controller  was  not  aware  that  an 
OE  was  about  to  occur  or  the  clearance  information  was 
not  contained  in  the  narrative,  all  altitude  and  heading 
changes  occurring  after  the  conflict  alert  warning  (i.e., 
the  alert  just  prior  to  the  loss  of  separation)  were  marked 
for  exclusion.  The  exclusions  were  then  subtracted  from 
the  summary  measures  (i.e.,  Number  of  Transitioning 
Aircraft  and  Number  of  Heading  Changes).  Altitude 
changes  were  evaluated  by  aircraft  rather  than  tabulated 


changes.  The  value  of  the  Number  of  Transitioning  Air¬ 
craft  remained  unaltered  if  an  aircraft  with  one  excluded 
altitude  change  had  other  changes  not  associated  with  an 
OE.  If  an  aircraft  made  only  one  change  or  all  altitude 
changes  were  associated  with  the  OE,  then  the  aircraft 
was  subtracted  from  the  total  value  of  the  Number  of 
Transitioning  Aircraft. 

Results 

Preliminary  Logistic  Regression  Analysis 

Stepwise  elimination  was  employed  because  such 
methods  are  extremely  valuable  in  the  exploratory  stages 
of  research  when  the  focus  is  on  identifying  predictors. 
Backward  elimination  was  used  for  selection  because  it  is 
less  prone  to  omit  useful  variables,  since  all  variables  are 
in  the  model  at  the  beginning  of  the  process  (Menard, 
1995).  The  likelihood-ratio  test,  which  compares  the  fit  of 
the  model  with  and  without  each  predictor  at  every  step, 
was  the  selection  criterion  used  because  it  is  more  rigor¬ 
ous  than  other  methods  (Norusis,  1990;  Pampel,  2000; 
Tabachnick  &  Fidell,  2006).  Hosmer  and  Lemeshow 
(2000)  recommend  using  criterion  levels  as  high  as .  1 5  or 
.20  to  ensure  that  all  relevant  variables  are  included  in  the 
logistic  regression  model.  However,  a  more  conservative 
criterion  level  of  .10  was  used  in  this  analysis. 

Descriptive  statistics  for  the  model-building  sample 
are  shown  in  Table  1 .  Descriptive  statistics  for  the  cross- 
validation  sample  are  shown  in  Table  2.  It  is  clear  that 
several  of  the  variables  were  not  normally  distributed. 
In  the  model-building  sample,  the  distribution  of  the 
Number  of  Point  Outs  differed  from  normal  by  nearly 
10  standard  deviations  in  skewness  and  14  standard  de¬ 
viations  in  kurtosis.  The  Number  of  Intersecting  Flight 
Paths  deviated  by  14  standard  deviations  in  skewness 
and  by  more  than  40  standard  deviations  in  kurtosis.  In 
the  cross-validation  sample,  departures  from  normality 
in  skewness  ranged  from  13  (Average  Control  Duration) 
to  more  than  91  (Number  of  Point  Outs)  standard  de¬ 
viations.  Departures  from  normality  in  kurtosis  ranged 
from  just  under  4  (Number  of  Transitioning  Aircraft) 


Table  1 .  Descriptive  Statistics:  Model-building  Sample  (N  =  96J 


Variable 

Mean 

SD 

Skew.1 

Kurtosis 

Average  Control  Duration  (seconds) 

182.05 

50.91 

-1.19 

2.09 

Number  of  Handoffs 

4.54 

3.05 

.72 

.02 

Number  of  Heading  Changes 

2.19 

2.19 

1.27 

1.28 

Number  of  Intersecting  Flight  Paths 

.56 

.97 

3.54 

19.67 

Number  of  Point  Outs 

1.13 

1.76 

2.44 

6.67 

Number  of  Transitioning  Aircraft 

3.54 

2.29 

.54 

-.27 

1 SE  Skewness  =  .246;  2SE  Kurtosis  =  .488 
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Table  2.  Descriptive  Statistics:  Cross-validation  Sample  (N  =  2667) 


Variable 

Mean 

SD 

Skew.1 

Kurtosis 

Average  Control  Duration  (seconds) 

173.13 

53.60 

-.63 

.90 

Number  of  Handoffs 

3.75 

2.70 

1.01 

1.82 

Number  of  Heading  Changes 

1.45 

1.57 

1.58 

3.77 

Number  of  Intersecting  Flight  Paths 

.26 

.52 

2.10 

5.18 

Number  of  Point  Outs 

.87 

1.59 

4.32 

34.39 

Number  of  Transitioning  Aircraft 

2.95 

2.23 

.80 

.36 

fSE  Skewness  =  .047;  2SE  Kurtosis  =  .095 


Table  3.  Correlation *  Matrix:  Model-building  Sample  (N  -  96) 


1 

2 

3 

4  5 

1 

Average  Control  Duration 

2 

Number  of  Handoffs 

-.29** 

3 

Number  of  Heading  Changes 

.23* 

.33** 

4 

Number  of  Intersecting  Flight  Paths 

.40** 

.26* 

.56** 

5 

Number  of  PointOuts 

.07 

.08 

.22 

.17 

6 

Number  of  Transitioning  Aircraft 

.03 

.38** 

.58** 

.38**  .22* 

*  Spearman’s  rho;  **p  <  .01 ;  *p  <  .05 


Table  4.  Correlation *  Matrix:  Cross-validation  Sample  (N  =  2667) 


1 

2 

3 

4  5 

1 

Average  Control  Duration 

2 

Number  of  Handoffs 

-.10** 

3 

Number  of  Heading  Changes 

.26** 

.29** 

4 

Number  of  Intersecting  Flight  Paths 

.27** 

.26** 

.33** 

5 

Number  of  Point  Outs 

.06** 

.23** 

.26** 

19** 

6 

Number  of  Transitioning  Aircraft 

.13** 

.42** 

.45** 

33**  23** 

*  Spearman’s  rho;  **p  <  .01 ;  *p  <  .05 


to  more  than  362  (Number  of  Point  Outs)  standard 
deviations.  Beyond  theoretical  issues  surrounding  the  use 
of  frequency  data  in  parametric  statistics,  such  extreme 
departures  from  normality  confirm  the  choice  of  a  non- 
parametric  statistic. 

Though  logistic  regression  is  a  “distribution-free” 
statistic,  it  is  not  assumption  free.  As  with  other  forms 
of  regression,  multicollinearity  among  the  predictors 
can  lead  to  biased  estimates  (Menard,  1995).  None  of 
the  Spearman’s  correlation  coefficients  shown  in  Table  3 
are  of  sufficient  magnitude  to  suggest  multicollinearity 
in  the  predictor  set.  Tolerance  values  were  >.45  for  all 
predictors,  far  in  excess  of  the  <.20  that  would  indicate 
multicollinearity.  Correlations  of  the  predictors  in  the 
cross-validation  sample  are  shown  in  Table  4.  Although 
all  of  the  coefficients  are  statistically  significant  (no  doubt 
due  to  the  size  of  the  sample),  the  pattern  of  associations 
is  similar  to  that  of  the  model-building  sample. 


The  Model  x2  represents  the  difference  between  the 
-2  log  likelihood  for  the  model  containing  the  predictor 
variables  and  that  of  the  model  with  the  constant  only. 
Thus,  the  Model  x2  is  analogous  to  the  overall  Ain  linear 
regression  because  it  tests  the  null  hypothesis  that  the  coef¬ 
ficients  for  all  predictors  in  the  model  equal  zero  (Norusis, 
1990).  The  final  logistic  regression  model  for  this  sample 
generated  a  Model  %2(3,  N=96 )  =  51.67,  p  <.01,  indi¬ 
cating  significantly  improved  prediction  over  the  model 
with  the  constant  only.  The  Hosmer-LemeshowTest  is  a 
particularly  robust  measure  of  fit.  Cases  are  divided  into 
deciles  of  predicted  probabilities,  and  then  observed  and 
expected  probabilities  are  compared  within  each  decile. 
As  with  most  goodness-of-fit  measures,  a  non-significant 
result  is  desirable.  The  Hosmer-Lemeshow  x2  (7,  N=96) 
=  6.10 ,  p  =.53  for  this  sample  indicated  that  the  model 
fit  the  data  reasonably  well. 
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Table  5.  Logistic  Regression  Summary  (N  =  96) 


Variable 

B 

S.E. 

Odds 

95%  Cl 

P 

Number  of  Intersecting  Flight  Paths 

1.60 

.50 

4.94 

1.86 

13.12 

.000 

Number  of  Heading  Changes 

.46 

.21 

1.58 

1.05 

2.37 

.019 

Number  of  Handoffs 

.25 

.11 

1.28 

1.03 

1.59 

.019 

Constant 

-3.10 

.71 

.05 

Table  6.  Classification:  Model-building  Sample  (N 

=  96) 

Predicted 


Routine  Operation 

Operational  Error 

Total 

Routine  Operation 

41  (85%) 

7  (15%) 

48 

Operational  Error 

14  (29%) 

34  (71%) 

48 

Table  7.  Classification:  Cross-validation  Sample  (N  =  2667) 


T 3 

CD 

in 

.Q 

O 


Routine  Operation 
Operational  Error 


P  redicted 

Routine  Operation  Operational  Error  Total 


1829  (71%) 

767  (29%) 

2596 

20  (28%) 

51  (72%) 

71 

Logistic  regression  coefficients  ( B ),  standard  errors, 
estimated  odds  ratios  (Odds),  95%  confidence  intervals 
for  the  odds  ratios  (95%  Cl),  and  significance  values 
for  the  likelihood- ratio  tests  are  provided  in  Table  5. 
Note  that  neither  the  logistic  regression  coefficients 
nor  standard  errors  are  inflated.  The  absence  of  large 
coefficients  indicates  a  sufficient  ratio  of  cases  to  predic¬ 
tors.  Variables  included  in  the  model  were  the  Number 
of  Intersecting  Flight  Paths,  the  Number  of  Heading 
Changes,  and  the  Number  of  Handoffs.  In  this  sample, 
every  one-unit  increase  in  the  Number  of  Intersecting 
Flight  Paths  increased  the  likelihood  by  a  multiplicative 
factor  of  4.94  (i.e.,  increased  the  odds  by  e4'94).  In  other 
words,  each  intersecting  flight  path  detected  increased  the 
likelihood  that  the  traffic  sample  was  an  OE  by  394%. 
Every  heading  change  increased  the  likelihood  by  58%, 
and  each  handoff  increased  the  likelihood  by  28%  that 
a  traffic  sample  was  an  OE. 

Classification  accuracy  in  the  model-building  sample  is 
shown  in  Table  6 .  In  this  sample  ,41  (85%)ofthe48  ROs 
were  correctly  classified,  and  7  (15%)  were  misclassified 
as  OEs.  Of  the  48  OEs  in  the  model-building  sample, 
34  (71%)  were  correctly  classified  and  14  (29%)  were 
misclassified  as  ROs.  Overall,  classification  reached  78% 
accuracy  in  the  model-building  sample.  This  represents 


28%  improvement  over  prior  probabilities  (i.e.,  the  num¬ 
ber  that  would  be  correctly  classified  by  chance). 

Classification  accuracy  in  the  cross-validation  sample 
is  shown  in  Table  7.  Keep  in  mind  that  relative  group  size 
does  not  matter  in  the  cross-validation  sample  because 
these  cases  are  not  being  used  to  build  the  model.  The 
existing  model  is  simply  being  applied  to  the  sample. 
Of  the  ROs  in  the  cross-validation  sample,  1829  (71%) 
were  correctly  classified  and  767  (29%)  were  misclassi¬ 
fied  as  OEs.  Of  the  OEs  in  the  cross-validation  sample, 
51  (72%)  were  correctly  classified  and  20  (28%)  were 
misclassified  as  ROs.  Overall  classification  accuracy  was 
71%  in  the  cross-validation  sample. 

Although  the  loss  of  classification  accuracy  when  the 
model  was  applied  to  the  RO  samples  was  somewhat 
disappointing,  the  question  of  whether  the  samples 
were  similar  enough  for  pooling  applied  only  to  the  OE 
samples.  The  similarity  between  classification  accuracy 
for  the  model-building  (7 1  %)  and  cross-validation  (72%) 
OE  samples  suggested  they  were  homogenous  enough  to 
justify  pooling  for  separate  logistic  regression  analyses  of 
the  low-  and  high-altitude  sectors. 

In  the  pooled  sample  of  OEs,  40  occurred  in  the 
low-altitude  sectors  and  79  occurred  in  the  high-altitude 
sectors.  The  40  low-altitude  OE  traffic  samples  were 
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combined  with  40  randomly-selected  low- altitude  RO 
traffic  samples  to  produce  a  total  of  80  traffic  samples  for 
the  low- altitude  sector  analysis.  The  79  high-altitude  OE 
traffic  samples  were  combined  with  7 9  randomly-selected 
high-altitude  RO  traffic  samples  to  produce  a  total  of  1 58 
traffic  samples  for  the  high-altitude  sector  analysis.  Back¬ 
ward  stepwise  elimination  using  the  likelihood-ratio  test 
as  selection  criterion  (p  =.10)  was  employed  for  both  the 
low-  and  high-altitude  sector  logistic  regression  analyses. 
Appendix  A  contains  a  list  of  the  ZID  sectors  and  the 
number  of  low-altitude  (Table  A2)  and  high-altitude 
(Table  A3)  traffic  samples  associated  with  each. 

Logistic  Regression  Analysis:  Low-Altitude  Sectors 

As  shown  inTable  8,  the  Number  ofHeading  Changes 
is  highly  correlated  with  both  the  Number  of  Intersecting 
Flight  Paths  (r=.6 2)  and  the  Number  of  Transitioning 
Aircraft  (r =.68) .  The  association  between  Number  of 
Transitioning  Aircraft  and  the  Number  of  Handoffs 
(r=.6l)  is  also  of  sufficient  magnitude  for  concern. 
Nevertheless,  Tolerance  values  were  >  .40  for  all  predic¬ 
tors,  indicating  there  is  no  reason  to  suspect  bias  due  to 
multicollinearity  in  the  low- altitude  sector  sample. 

The  logistic  regression  model  for  the  low-altitude 
sample  generated  a  Model  %2(3,  A^=  80)  =  23.82,  p  <.01, 
indicating  significantly  improved  prediction  over  the 
model  with  the  constant  only.  The  non-significant 
Hosmer-Lemeshow  yp  (8,  AMO)  =  1.61,  p=. 99  for  the 
low-altitude  sample  signifies  that  the  model  fit  the  data 
well.  Logistic  regression  coefficients  ( B ),  standard  errors, 
estimated  odds  ratios  (Odds),  95%  confidence  intervals 
for  the  odds  ratios  (95%  Cl),  and  significance  values 
for  the  likelihood-ratio  tests  for  the  low- altitude  sector 
sample  are  provided  in  Table  9.  As  with  the  full  sample, 
neither  the  logistic  regression  coefficients  nor  standard 
errors  are  inflated,  indicating  a  sufficient  ratio  of  cases 
to  predictors. 

In  the  low-altitude  sample  model,  the  Number  of  In¬ 
tersecting  Flight  Paths  had  the  highest  odds  ratio  (2.89), 
followed  by  the  Number  of  Point  Outs  (1.57),  and  the 
Number  of  Handoffs  (1.19).  In  other  words,  each  inter¬ 
secting  flight  path  increased  the  likelihood  that  the  traffic 


sample  was  an  OE  by  189%,  each  point  out  increased 
the  likelihood  by  57%,  and  each  handoff  increased  OE 
likelihood  by  1 9%.  However,  the  confidence  intervals  for 
the  Number  of  Point  Outs  and  the  Number  of  Handoffs 
suggest  that  the  parameter  estimates  for  these  predictors 
might  not  generalize  to  another  sample. 

Classification  accuracy  in  the  low-altitude  sample 
(Table  1 0)  was  similar  to  that  of  the  combined  sample. 
Of  the  40  ROs  in  the  low-altitude  sample,  32  (80%) 
were  correctly  classified  and  8  (20%)  were  misclassified 
as  OEs.  Of  the  40  OEs  in  the  sample,  28  (70%)  were 
correctly  classified  and  12  (30%)  were  misclassified  as 
ROs.  Overall,  the  low-altitude  model  had  75%  clas¬ 
sification  accuracy. 

Logistic  Regression  Analysis:  High-Altitude  Sectors 

Although  several  of  the  Spearman’s  correlations  shown 
inTable  1 1  are  statistically  significant,  none  of  coefficients 
are  of  sufficient  magnitude  to  suggest  multicollinearity. 
Accordingly,  Tolerance  values  were  high  (.56  and  above) 
for  all  predictors  in  this  sample. 

The  logistic  regression  model  for  the  high-altitude 
sample  generated  a  Model  y2(4,  77=158)  =  73.01,  p 
<.01,  indicating  significantly  improved  prediction  over 
the  model  with  the  constant  only.  The  non-significant 
Hosmer-Lemeshow  yp  (8,  AM  58)  =  3.33,  p=. 91  for 
the  high- altitude  sample  verified  that  the  model  fit  the 
data.  Logistic  regression  coefficients  ( B ),  standard  errors, 
estimated  odds  ratios  (Odds),  95%  confidence  intervals 
for  the  odds  ratios  (95%  Cl),  and  significance  values 
for  the  likelihood-ratio  tests  for  the  high-altitude  sector 
sample  are  provided  inTable  12.  As  with  the  full  sample, 
neither  the  logistic  regression  coefficients  nor  standard 
errors  are  inordinately  large,  indicating  a  sufficient  ratio 
of  cases  to  predictors. 

In  the  high-altitude  sample  model,  the  Number  of 
Intersecting  Flight  Paths  had  the  highest  odds  ratio 
(2.00),  followed  by  the  Number  of  Heading  Changes 
(1.36),  the  Number  of  Transitioning  Aircraft  (1.27),  and 
Average  Control  Duration  (1.01).  In  other  words,  each 
one-unit  increase  in  the  Number  of  Intersecting  Flight 
Paths  increased  the  likelihood  that  a  traffic  sample  was 


Table  8.  Correlation*  Matrix:  Low-Altitude  Sectors  (N  =  80) 


1 

2 

3 

4  5 

1 

Average  Control  Duration 

2 

Number  of  Handoffs 

-.27* 

3 

Number  of  Heading  Changes 

.21 

.40** 

4 

Number  of  Intersecting  Flight  Paths 

.33** 

.32** 

.62** 

5 

Number  of  Point  Outs 

-.14 

.17 

.25* 

.06 

6 

Number  of  Transitioning  Aircraft 

.22 

.61** 

.68** 

.56**  .08 

*  Spearman's  rho;  **p  <  .01 ;  *p  <  .05 


Table  9.  Logistic  Regression  Summary:  Low-Altitude  Sectors  (N  =  80) 


Variable 

B 

S.E. 

Odds 

95%  Cl 

P 

Number  of  Intersecting  Flight  Paths 

1.06 

.36 

2.89 

1.43 

5.83 

.00 

Number  of  Point  Outs 

.45 

.24 

1.57 

.98 

2.52 

.04 

Number  of  Handoffs 

.17 

.11 

1.19 

.97 

1.46 

.10 

Constant 

-1.94 

.57 

.14 

Table  10.  Classification:  Low-Altitude  Sectors  (N  =  80) 

Predicted 


Routine  Operation 

Operational  Error 

Total 

Routine  Operation 

32  (80%) 

8  (20%) 

40 

Operational  Error 

12  (30%) 

28  (70%) 

40 

Table  1 1 .  Correlation *  Matrix:  High-Altitude  Sectors  (N  -  158) 


1 

2 

3 

4  5 

1 

Average  Control  Duration 

2 

Number  of  Handoffs 

-.14 

3 

Number  of  Heading  Changes 

.36** 

.42** 

4 

Number  of  Intersecting  Flight  Paths 

.49** 

.22** 

.51** 

5 

Number  of  Point  Outs 

.11 

.24** 

.24** 

6 

Number  of  Transitioning  Aircraft 

.16* 

.43** 

.53** 

.39**  .19* 

*  Spearman’s  rho;  **p  <  .01 ;  *p  <  .05 


Table  12.  Logistic  Regression  Summary:  High-Altitude  Sectors  (N  =  758J 


Variable 

B 

S.E. 

Odds 

95%  Cl 

P 

Number  of  Intersecting  Flight  Paths 

.69 

.28 

2.00 

1.16 

3.45 

.01 

Number  of  Heading  Changes 

.31 

.15 

1.36 

1.01 

1.83 

.03 

Number  of  Transitioning  Aircraft 

.24 

.11 

1.27 

1.03 

1.57 

.02 

Average  Control  Duration 

.01 

.01 

1.01 

1.00 

1.03 

.01 

Constant 

-4.43 

1.11 

.01 
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Table  13.  Classification:  High-Altitude  Sectors  (N  =  158) 


T 3 

OJ 

in 

JD 

O 


Routine  Operation 
Operational  Error 


Predicted 

Routine  Operation  Operational  Error  Total 


64  (81%) 

15  (19%) 

79 

19  (24%) 

60  (76%) 

79 

an  OE  by  100%,  each  one-unit  increase  in  the  Number 
of  Heading  Changes  increased  the  likelihood  by  36%, 
every  Transitioning  Aircraft  increased  the  likelihood  by 
27%,  and  each  one-second  increase  in  Average  Control 
Duration  increased  the  likelihood  by  1%. 

Classification  accuracy  in  the  high-altitude  sample 
(Table  1 3)  was  slightly  better  than  that  of  the  low- altitude 
sample.  Of  the  79  ROs  in  the  high-altitude  sample,  64 
(81%)  were  correctly  classified  and  15  (19%)  were  mis- 
classified  as  OEs.  Of  the  79  OEs  in  the  sample,  60  (76%) 
were  correctly  classified,  and  1 9  (24%)  were  misclassified 
as  ROs.  Overall,  the  high- altitude  model  had  79%  clas¬ 
sification  accuracy.  This  represents  a  29%  improvement 
over  prior  probabilities  (i.e.,  the  number  that  would  be 
correctly  classified  by  chance). 

Discussion 

The  results  of  the  logistic  regression  analyses  indicate 
that  a  sufficient  model  may  be  constructed  from  sector 
characteristics  variables.  Overall  classification  accuracy  be¬ 
tween  75-7 9%  is  remarkable  for  models  constructed  solely 
of  environmental  and  contextual  factors.  After  all,  other 
factors  (e.g.,  human  elements,  organizational  influences) 
also  contribute  to  the  occurrence  of  OEs.  Unfortunately, 
all  the  logistic  regression  models  were  better  at  classifying 
ROs  than  OEs.  Classification  of  OEs  ranged  from  as  low 
as  70%  in  the  low- altitude  sector  sample  to  76%  in  the 
high-altitude  sample.  Although  this  level  of  accuracy 
would  be  unacceptable  for  most  automation  tools,  it  is 
unrealistic  to  expect  definitive  results  from  one  or  two 
analyses.  Moreover,  the  sector  characteristic  variables  used 
in  these  analyses  do  not  represent  an  exhaustive  list  of  all 
the  potential  predictors  of  OEs. 

One  of  the  most  unexpected  findings  was  the  unique¬ 
ness  of  the  preliminary  analysis  model.  The  Number  of 
Handoffs  had  an  odds  ratio  of  1 .28,  and  yet  this  variable 
failed  to  demonstrate  a  similar  level  of  influence  in  the 
low- altitude  model  and  was  conspicuously  absent  from  the 
high- altitude  model.  The  relative  influence  of  the  Number 
of  Handoffs  when  low-  and  high- altitude  sectors  were 
combined  is  either  an  indictment  of  the  use  of  regression 


techniques  in  general  (i.e.,  they  tend  to  capitalize  on  the 
unique  characteristics  of  the  sample)  or  an  indication 
that  traffic  count  becomes  more  salient  as  the  level  of 
analysis  changes.  Most  dynamic  variables  correlate,  to 
a  greater  or  lesser  degree,  with  traffic  count.  Therefore, 
traffic  count  may  emerge  as  unique  characteristics  of  the 
low-  and  high-altitude  sectors  are  obscured.  That  traffic 
count  remains  the  single  best  predictor  of  OEs  at  the 
national  level,  yet  fails  to  predict  well  at  the  sector  level, 
may  be  an  example  of  this  phenomenon.  Consequently, 
differences  between  the  low-altitude,  high-altitude,  and 
combined  models  might  have  implications  for  the  suit¬ 
ability  of  applying  policies  based  on  evaluations  made  at 
the  national  level  at  the  sector  level. 

Low-Altitude  Sector  Model 

The  most  influential  variable  in  the  low-altitude 
sector  model  was  the  Number  of  Intersecting  Flight 
Paths  (Odds  =  2.89),  followed  by  the  Number  of  Point 
Outs,  and  the  Number  of  Handoffs  (Odds  =  1.19). 
Aside  from  the  addition  of  the  Number  of  Intersecting 
Flight  Paths,  this  model  was  similar  to  the  low-altitude 
sector  model  in  Pfleiderer  and  Manning  (2007).  In  the 
previous  study,  the  most  influential  predictor  was  the 
Number  of  Point  Outs  (Odds  =  3.30),  followed  by  the 
Number  of  Handoffs  (Odds  =  1.54),  and  the  Number 
of  Heading  Changes  (Odds  =  1.49).  Doubts  about  the 
validity  of  the  Number  of  Heading  Changes  as  a  genuine 
predictor  in  the  previous  study  were  due  to  concerns  that 
much  of  its  effect  in  the  low-altitude  model  was  related 
to  clearances  made  in  an  attempt  to  resolve  the  OE.  As 
anticipated,  the  Number  of  Heading  Changes  was  not 
included  in  the  low- altitude  sector  model  once  this  vari¬ 
able  was  adjusted.  Both  the  Number  of  Point  Outs  and 
the  Number  of  Handoffs  lost  a  considerable  amount  of 
reliability,  as  evidenced  by  the  range  of  the  confidence 
intervals.  On  the  other  hand,  the  fact  that  the  Number 
of  Point  Outs  and  the  Number  of  Handoffs  were  also 
included  in  the  previous  model  (based  on  a  completely 
different  sample  of  ROs)  somewhat  belies  the  contention 
that  these  parameter  estimates  might  not  generalize  to 
another  sample. 
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On  the  whole,  classification  accuracy  in  the  low- alti¬ 
tude  samples  was  inferior  to  that  of  the  previous  study. 
Although  classification  of  the  RO  samples  was  improved 
(80%  as  opposed  to  77%  in  the  previous  analysis),  OE 
classification  was  not.  The  current  model,  as  compared 
to  82%  accurate  classification  in  the  previous  analysis, 
accurately  classified  only  70%  of  the  OE  samples. 

The  predictive  strength  of  the  Number  of  Point  Outs 
and  the  Number  of  ffandoffs  in  the  Pfleiderer  and  Man¬ 
ning  (2007)  results  suggested  that  coordination  played 
a  primary  role  in  the  development  of  OEs  in  the  ZID 
low-altitude  sectors.  This  impression  was  bolstered  by 
the  Pfleiderer  et  al.  (2007)  data,  in  which  controllers 
and  supervisors  at  ZID  rated  coordination  as  one  of  the 
primary  sources  of  complexity  in  low-altitude  sectors. 
Consequently,  the  emergence  of  the  Number  of  Inter¬ 
secting  Flight  Paths  as  the  most  influential  predictor  in 
the  current  low-altitude  logistic  regression  model  was 
surprising,  because  controller  and  supervisor  ratings  for 
this  complexity  factor  were  moderate  in  the  low- altitude 
sectors.  The  results  of  the  logistic  regression  analysis 
suggest  that  coordination  may  be  a  contributing  fac¬ 
tor,  but  converging  traffic  patterns  might  be  of  greater 
consequence. 

High-Altitude  Sector  Model 

The  Number  of  Intersecting  Flight  Paths  was  the  most 
influential  predictor  in  the  high-altitude  sector  model 
(Odds  =  2.00),  followed  by  the  Number  of  Heading 
Changes.  As  anticipated,  there  was  a  reduction  in  the 
relative  influence  of  the  Number  of  Heading  Changes 
after  adjustments  were  made  to  exclude  changes  made 
in  response  to  clearances  to  avoid  an  imminent  OE  (i.e., 
the  estimated  odds  ratio  was  reduced  from  2.28  in  the 
previous  study  to  1 . 36  in  the  present  study) .  Nevertheless, 
it  remained  a  significant  predictor  in  the  high-altitude 
model.  Other  elements  of  the  model  were  nearly  identical 
to  those  of  Pfleiderer  and  Manning  (2007).  The  Number 
of  Transitioning  Aircraft  had  an  estimated  odds  ratio  of 
1.26  in  the  previous  study  and  increased  to  1.27  in  the 
present  one.  Average  Control  Duration  had  an  estimated 
odds  ratio  of  1.02  in  the  previous  study  and  decreased 
to  1.01  in  the  present  one.  Classification  accuracy  in  the 
high-altitude  sample  was  also  relatively  consistent  between 
the  two  studies.  Both  models  correctly  classified  81%  of 
the  ROs.  However,  79%  of  the  OEs  were  correctly  clas¬ 
sified  in  the  previous  analysis,  whereas  only  76%  were 
correctly  classified  in  this  one. 

The  influence  of  the  Number  of  Intersecting  Flight 
Paths  was  no  surprise  in  this  sample,  because  ZID  control¬ 
lers  and  supervisors  rated  this  complexity  factor  one  of 
the  most  influential  in  the  high-  and  super  high- altitude 


sectors  (Pfleiderer  et  al.,  2007).  The  Number  of  Heading 
Changes  remained  influential,  despite  changes  to  the 
processing  interval  and  adjustments  to  the  variable  itself. 
This  is  consistent  with  Laudeman  et  al.  ( 1 998) ,  in  which 
heading  changes  received  the  highest  beta  weight  in  a 
linear  multiple  regression  analysis  of  controller  ratings  of 
activity  in  four  sectors  at  the  Denver  ARTCC.  In  their 
discussion,  the  authors  attributed  the  influence  ofheading 
changes  to  the  “significant  arrival  traffic  in  all  the  sectors 
that  were  observed”  (p.  7).  Arrival  and  departure  traffic 
complexity  is  generally  considered  to  be  a  low- altitude 
phenomenon,  but  this  perception  may  be  inaccurate.  In 
the  present  study,  the  Number  of  Heading  Changes  was 
extremely  influential  in  the  high- altitude  model  but  failed 
to  be  included  in  the  low- altitude  sector  model. 

The  third  most  influential  factor  in  the  high- altitude 
logistic  regression  analysis  was  the  Number  ofTransition- 
ing  Aircraft.  Climbing  and  descending  traffic  has  long  been 
recognized  as  a  contributor  to  the  difficulty  of  working 
a  sector  (e.g.,  Arad,  1964;  Grossberg,  1989;  Kopardekar 
&  Magyarits,  2003).  This  finding  is  also  consistent  with 
Pfleiderer  et  al.  (2007),  in  which  the  complexity  factor 
Climbing  and  Descending  Traffic  received  the  highest 
complexity  rating  for  the  high-and  super  high-altitude 
sectors. 

The  fourth  variable  included  in  the  high-altitude 
sample  model  was  Average  Control  Duration.  Odds  ra¬ 
tios  in  logistic  regression  are  an  indication  of  effect  size. 
The  closer  the  odds  are  to  zero,  the  smaller  the  effect 
size.  Consequently,  a  1.01  odds  ratio  suggests  that  the 
effect  for  Average  Control  Duration  is  small.  However, 
it  is  important  to  remember  that  logistic  regression 
coefficients  (i.e.,  the  natural  logs  of  the  odds  ratios)  are 
not  standardized.  Average  Control  Duration  is  based  on 
the  number  of  seconds  each  aircraft  was  controlled  by 
the  sector.  Thus,  a  minimal  change  in  Average  Control 
Duration  produced  a  relatively  large  1%  change  in  the 
likelihood  that  a  traffic  sample  was  an  OE. 

Future  Research 

Although  logistic  regression  cannot  be  used  to  identify 
causal  factors  directly  (i.e.,  prediction  is  not  the  same  as 
causation),  the  logistic  regression  coefficients  do  provide 
information  about  the  likelihood  of  an  OE  relative  to  the 
predictors  in  the  model.  Thus,  the  results  have  immedi¬ 
ate  heuristic  value  in  that  they  invite  questions  about 
how  the  dynamic  predictors  interact  with  static  sector 
characteristics.  Dynamic  elements  lend  themselves  to 
automation  applications,  but  static  characteristics  must  be 
addressed  by  sector  restructuring.  The  dynamic  predictors 
that  make  up  the  logistic  regression  models  are  indicants 
of  conditions  that  discriminate  between  OEs  and  ROs. 
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These  indicants  may  be  used  to  reveal  aspects  of  the  sector 
environment  that  might  be  altered  to  reduce  the  number 
of  OEs.  For  example,  the  combination  of  the  Number 
of  Point  Outs  and  the  Number  of  Handoffs  in  the  low- 
altitude  sector  model  may  indicate  that  the  location  of 
sector  boundaries  increases  coordination  workload  and 
complexity.  As  Couluris  and  Schmidt  (1973)  observed, 
handoffs  and  point  outs  “result  from,  or  are  influenced  by, 
the  existence  and  design  (shape)  of  the  sectors”  (p.  657). 
On  the  other  hand,  the  combination  of  the  Number  of 
Point  Outs  and  the  Number  of  Intersecting  Flight  Paths 
may  point  to  problems  with  the  orientation  of  traffic 
paths  relative  to  those  boundaries.  The  combination 
of  the  Number  of  Heading  Changes  and  the  Number 
of  Transitioning  Aircraft  in  the  high- altitude  sectors  is 
suggestive  of  traffic  complexity  in  high-altitude  sectors 
adjacent  to  low-altitude  arrival  or  departure  sectors.  Av¬ 
erage  Control  Duration  as  a  predictor  of  OEs  may  be  a 
function  of  the  size  of  high- altitude  sectors. 

Because  of  the  research  that  remains  to  be  accomplished, 
these  results  must  be  viewed  as  preliminary.  Given  the 
sample  size  and  consequent  restriction  of  the  predictor 
set,  there  is  no  guarantee  that  these  results  will  generalize 
to  other  samples.  Multiple  studies,  with  samples  sizes  that 
allow  for  a  more  inclusive  list  of  predictors,  must  be  con¬ 
ducted  at  a  number  of  facilities  before  such  models  might 
be  reliable  enough  forpractical  applications.  Nevertheless, 
the  methodology  of  comparing  OE  and  RO  traffic  samples 
is  promising.  Continued  investigations  along  these  lines 
may  highlight  complexity  factors  that  must  be  addressed 
to  ensure  that  separation  is  maintained. 
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Appendix  A 

Indianapolis  Air  Route  Traffic  Control  Center  Traffic  Samples 


Table  A1 .  Model-building  and  Cross-validation  Samples 


Sector 

Number 

Sector 

Name 

Sector 

Strata 

Number  of  Associated  Traffic  Samples 

Model-building  Sample 

Cross-validation  Sample 

Routine 

Operation 

Operational 

Error 

Routine 

Operation 

Operational 

Error 

18 

Nabb 

LO 

2 

3 

71 

4 

19 

New  Hope 

LO 

1 

2 

75 

1 

20 

Lexington 

LO 

2 

3 

43 

1 

21 

London 

LO 

3 

0 

73 

1 

24 

Parkersburg 

LO 

0 

0 

76 

1 

25 

Hazard 

LO 

0 

1 

60 

0 

26 

River 

LO 

1 

1 

74 

1 

30 

Columbus 

LO 

0 

2 

76 

2 

31 

Lytle 

LO 

0 

1 

75 

3 

32 

Rosewood 

LO 

0 

1 

70 

1 

33 

Muncie 

LO 

3 

0 

73 

1 

34 

Shelbyville 

LO 

3 

1 

73 

3 

35 

Terre  Haute 

LO 

0 

0 

62 

3 

66 

Madison 

HI 

1 

3 

75 

3 

69 

Pike 

LO 

2 

1 

74 

2 

76 

Batesville 

SH 

2 

0 

63 

1 

78 

Springfield 

HI  (IM) 

1 

1 

75 

1 

79 

Bobcat 

HI  (IH) 

0 

4 

73 

1 

80 

King 

HI 

1 

1 

73 

2 

81 

Pocket  City 

HI 

3 

1 

72 

3 

82 

Louisville 

HI 

2 

3 

72 

5 

83 

Falmouth 

HI 

3 

1 

74 

1 

84 

Rebel 

HI 

2 

0 

74 

0 

85 

Charleston 

HI 

2 

1 

74 

4 

86 

Beckley 

HI 

1 

3 

75 

2 

87 

Appleton 

HI 

3 

3 

73 

2 

88 

Dayton 

HI 

2 

2 

72 

6 

89 

Indianapolis 

HI 

1 

2 

75 

5 

91 

Impel 

SH 

1 

2 

62 

1 

92 

Mystic 

SH 

1 

1 

51 

2 

93 

Dacos 

SH 

1 

0 

75 

2 

94 

Somerset 

SH 

0 

0 

74 

2 

95 

Henderson 

SH 

0 

2 

76 

1 

96 

Bluefield 

SH 

0 

1 

74 

0 

97 

Lockbourne 

SH 

3 

0 

73 

1 

98 

Patterson 

SH 

1 

0 

48 

0 

99 

Wabash 

SH 

0 

1 

68 

2 

TOTAL 

48 

48 

2596 

71 

At 


Table  A2.  Low-Altitude  Sector  Sample 


Sector 

Number 

Sector 

Name 

Sector 

Number  of  Associated  Traffic  Samples 

Strata 

Routine  Operation 

Operational  Error 

18 

Nabb 

LO 

0 

7 

19 

New  Hope 

LO 

4 

3 

20 

Lexington 

LO 

1 

4 

21 

London 

LO 

2 

1 

24 

Parkersburg 

LO 

4 

1 

25 

Hazard 

LO 

2 

1 

26 

River 

LO 

2 

2 

30 

Columbus 

LO 

3 

4 

31 

Lytle 

LO 

4 

4 

32 

Rosewood 

LO 

3 

2 

33 

Muncie 

LO 

0 

1 

34 

Shelbyville 

LO 

2 

4 

35 

Terre  Haute 

LO 

6 

3 

69 

Pike 

LO 

7 

3 

TOTAL 

40 

40 

Table  A3.  High-Altitude  Sector  Sample 


Sector 

Number 

Sector 

Name 

Sector 

Number  of  Associated  Traffic  Samples 

Strata 

Routine  Operation 

Operational  Error 

66 

Madison 

HI 

6 

6 

76 

Batesville 

SH 

3 

1 

78 

Springfield 

HI  (IM) 

5 

2 

79 

Bobcat 

HI  (IH) 

5 

5 

80 

King 

HI 

4 

3 

81 

Pocket  City 

HI 

1 

4 

82 

Louisville 

HI 

2 

8 

83 

Falmouth 

HI 

1 

2 

84 

Rebel 

HI 

4 

0 

85 

Charleston 

HI 

1 

5 

86 

Beckley 

HI 

1 

5 

87 

Appleton 

HI 

4 

5 

88 

Dayton 

HI 

3 

8 

89 

Indianapolis 

HI 

2 

7 

91 

Impel 

SH 

2 

3 

92 

Mystic 

SH 

3 

3 

93 

Dacos 

SH 

6 

2 

94 

Somerset 

SH 

3 

2 

95 

Henderson 

SH 

6 

3 

96 

Bluefield 

SH 

7 

1 

97 

Lockbourne 

SH 

5 

1 

98 

Patterson 

SH 

1 

0 

99 

Wabash 

SH 

4 

3 

TOTAL 

79 

79 

A 2 


